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Abstract 

This paper presents a novel selective constraint 
propagation method for constrained image segmen¬ 
tation. In the literature, many pairwise constraint 
propagation methods have been developed to ex¬ 
ploit pairwise constraints for cluster analysis. How¬ 
ever, since most of these methods have a polyno¬ 
mial time complexity, they are not much suitable 
for segmentation of images even with a moderate 
size, which is actually equivalent to cluster anal¬ 
ysis with a large data size. Considering the local 
homogeneousness of a natural image, we choose to 
perform pairwise constraint propagation only over 
a selected subset of pixels, but not over the whole 
image. Such a selective constraint propagation 
problem is then solved by an efficient graph-based 
learning algorithm. To further speed up our selec¬ 
tive constraint propagation, we also discard those 
less important propagated constraints during graph- 
based learning. Finally, the selectively propagated 
constraints are exploited based on Li-minimization 
for normalized cuts over the whole image. The ex¬ 
perimental results demonstrate the promising per¬ 
formance of the proposed method for segmentation 
with selectively propagated constraints. 


1 Introduction 

Image segmentation is a fundamental problem 
in computer vision. Despite many years of re- 


search I Shi and Malik, 2000 

[Martin a/., 2001 

Carson et al, 2002[ 

Unnikrishnan et al, 20071 


Arbelaez et al, 201 ll , general-purpose image segmen¬ 


tation is still a very challenging task because segmentation 
is inherently ill-posed. To improve the results of image 
segmentation, much attention has been paid to constrained 
image segmenta tion |Yu and Shi, 2004[|Eriksson et al, 2007t 
|Xu et al, 2009t [Ghanem and Ahuja, 2010| , where certain 
constraints are initially provided for image segmentation. In 
this paper, we focus on constrained image segmentation using 
pairwise constraints, which can be derived from the initial 
labels of selected pixels. In general, there exist two types of 
pairwise constraints: must-link constraint denotes a pair of 
pixels belonging to the same image region, while cannot-link 


constraint denotes a pair of pixels belonging to two different 
image regions. In previous work jWagstaff et al , 2()0T| 
[Klein et al, 2002t [Xing et al, 2003} [Hoi et al, 2006| , such 
weak supervisory information has been widely used to 
improve the performance of machine learning and pattern 
recognition IjLu, 2006t |Lu et al, 2008t |Lu and Ip, 2009} 
Lu and Peng, 201 1| in many challenging tasks. 


The main challenge in constrained image segmen¬ 
tation is how to effectively exploit a limited number 
of pairwise constraints for image segmentation. A 
sound solution is to perform pairwise constraint prop¬ 
agation to generate more pairwise constraints. Al¬ 
though many pairwise constraint prop agation meth¬ 
ods | Lu and Carreira-Perpina n, 2008} [Li et al, 20Q8i 
|Lu and Ip, 2QlQ[ ~ |Lu and Peng, 2Q13| have been devel¬ 
oped for const rained clustering I jKamvar et al , 2003) 
[Kulis et al, 2005| , they mostly have a polynomial time 
complexity and thus are not much suitable for segmentation 
of images even with a moderate size (e.g. 200 x 200 

pixels), which is actually equivalent to clustering with a 
large data size (i.e. = 40, 000). For constrained image 

segmentation, we need to develop more efficient pairwise 
constraint propagation met hods, instead of directly utilizing 
the existing m ethods like |Lu and Carreira-Perpinan, 2008t 
[Li et al, 2008[ |Lu and Ip, 2010t |Lu and Peng, 2013| . Here, 
it is worth noting that even the simple assignment operation 
incurs a large time cost of 0{N‘^) if we perform pairwise 
constraint propagation over all the pixels, since the number 
of all possible pairwise constraints is N{N — l)/2. The 
unique choice is to propagate the initial pairwise constraints 
only to a selected subset of pixels, but not across the whole 
image. 


Fortunately, the local homogeneousness of a natural image 
provides kind of supports for this choice, i.e., a selected sub¬ 
set of pixels may approximate the whole image (such down- 
sampling is widely used in image processing). More impor¬ 
tantly, the selectively propagated constraints over a selected 
subset of pixels are enough for achieving a good quality of 
image segmentation, as verified by our later experimental re¬ 
sults. Hence, in this paper, we develop a selective constraint 
propagation (SCP) method for constrained image segmenta¬ 
tion, whichpropagates the initial pairwise constraints only to 
a selected subset of pixels {the first meaning of our selective 
constraint propagation). Although there exist different sam- 


















































Our main contributions 



Figure 1: The flowchart of our selective constraint propagation (SCP) for constrained image segmentation, where the selectively 
propagated constraints are exploited based on Li-minimization for normalized cuts (NCuts). 


pling methods in statistics, we only consider random sam¬ 
pling for efficiency purposes, i.e., the subset of pixels used 
for selective constraint propagation are selected randomly 
from the whole image. In this paper, we formulate our selec¬ 
tive constraint propagation as a graph-based learning problem 
which can be efficiently solved based on the label propaga¬ 
tion technique | |Zhou et al, 20041 . To further speed up our 
algorithm, we also discard those less important propagated 
constraints during iteration of graph-based learning, which is 
the second meaning of our selective constraint propagation. 
To the best of our knowledge, we have made the first attempt 
to develop a selective constraint propagation method for con¬ 
strained image segmentation. 

Finally, the selectively propagated constraints obtained 
by our selective constraint propagation are exploited to ad¬ 
just the original weight matrix based on optimization tech¬ 
niques, in order to ensure that the new weight matrix is as 
consistent as possible with the selectively propagated con¬ 
straints. In this paper, we formulate such weight adjustment 
as an Li-minimiza tion problem ||Elad and Aharon, 2006[ 
IMairal et al ., 2008] [Wright et al ., 2009} |Xiao et al ., 201 Tf , 
which can be solved efficiently due to the limited num¬ 
ber of selectively propagated constraints. The obtained 
new weight matrix is then applied to normalized cuts 
I jShi and Malik, 20()0| for image segmentation. The flowchart 
of our selective constraint propagation for constrained image 
segmentation is illustrated in Figure [T] Although our selec¬ 
tive constraint propagation method is originally designed for 
constrained image segmentation, it can be readily extended 
to other challenging tasks (e.g., semantic image segmentation 
and multi-face tracking) where only a limited number of pair¬ 
wise constraints are provided initially. 


It should be noted that the present work is distinctly differ¬ 
ent from previous work on constrained image segmentation 


IjYu and Shi, 2004[ [Eriksson et al ., 20071 

Xu et al ., 2009[ 

Ghanem and Ahuj a, 201 0| . In (Yu and S 

li, 2004), only 


linear equality constraints (analogous to must-link con¬ 
straints) are exploited for image segmentation based on 


normalized cuts. In IjEriksson gr a/., 2007[ jXu et al ., 2009t 
Ghanem and Ahuja, 2010| , although more types of con¬ 
straints are exploited for image segmentation, the linear 
inequality constraints analogous to cannot-link constraints 
are completely ignored just as I jYu and Shi, 2004| |. In con¬ 
trast, our selective constraint propagation method exploits 


both must-link and cannot-link constraints for normalized 
cuts. More notably, as shown in our later experiments, our 
method obviously outperforms I jYu and Shi, 2004| | due to 
extra consideration of cannot-link constraints for image 
segmentation. 

The remainder of this paper is organized as follows. In Sec¬ 
tion 2, we develop a selective constraint propagation method 
which propagates the initial pairwise constraints only to a se¬ 
lected subset of pixels. In Section 3, the selectively prop¬ 
agated constraints are further exploited to adjust the original 
weight matrix based on Li-minimization for normalized cuts. 
Finally, Sections 4 and 5 give the experimental results and 
conclusions, respectively. 

2 Selective Constraint Propagation 

This section presents our selective constraint propagation 
(SCP) in detail. We first give our problem formulation for 
propagating the initial pairwise constraints only to a selected 
subset of pixels from a graph-based learning viewpoint, and 
then develop an efficient SCP algorithm based on the label 
propagation technique I jZhou et al, 2004| |. 

2.1 Problem Formulation 

In this paper, our goal is to propagate the initial pairwise con¬ 
straints to a selected subset of pixels for constrained image 
segmentation. To this end, we need to first select a subset 
of pixels for our selective constraint propagation. Although 
there exist different sampling methods in statistics, we only 
consider random sampling for efficiency purposes, i.e., the 
subset of pixels are selected randomly from the whole im¬ 
age. In the following, the problem formulation for selective 
constraint propagation over the selected subset of pixels is 
elaborated from a graph-based learning viewpoint. 

Let M = {(i, j) : h = Ij, I < i,j < N} denote the set 
of initial must-link constraints and C = : k ^ Ij,l < 

hi < N} denote the set of initial cannot-link constraints, 
where k is the region label assigned to pixel i and N is the 
total number of pixels within an image. The set of constrained 
pixels is thus denoted as Pc = • (hj) ^ Af U C, 1 < 

j < A^} U {i : (j, i)^MUC,l<j< A^} with Nc = 
\Pc\. Moreover, we randomly select a subset of pixels P^ C 
{1, 2,..., A^} with Ns = I Ps I, and then form the final selected 
subset of pixels used for our selective constraint propagation 
as Pn = Ps U Pc with Nu = \Pu\- 












































In this paper, we construct a /c-nearest neighbors (/c-NN) 
graph over all the pixels so that the normalized cuts for im¬ 
age segmentation can be performed efficiently over this k-NN 
graph. Let its weight matrix be ly = {w{i,j)}NxN- We 
define the weight matrix Wu = {wu{i,j)}NuxNu l^e 
selected subset of pixels Pu as: 

Wu{i,j) = w{Pu{i),Puij)), ( 1 ) 

where P„(i) denotes the i-th member of The normalized 
Laplacian matrix is then given by 

Lu = I - (2) 

where I is an identity matrix and D is a. diagonal matrix 
with its i-th diagonal entry being the sum of the i-th row of 
Wu. Moreover, for convenience, we represent the two sets 
of initial pairwise constraints A4 and C using a single matrix 
Zu = { 2 «(*,i)}iv„xiv„ as follows: 

r+1, (P„(i),P„(i)) GAf; 

Zu(i,j) = < -1, {Pu{i),Pu{j)) e C; (3) 

[O, otherwise. 

Based on the above notations, the problem of selective con¬ 
straint propagation over the selected subset of pixels Pu can 
be formulated from a graph-based learning viewpoint: 

min ||F„ - Zu\\f + IJ.tT{FjLuFv) + \\Fh - Zu\\f 

FvPh 

P^ti^PhLuP"^) j\\Fv — Fh\\‘F^ (4) 

where ji and 7 denote the positive regularization parame¬ 
ters, II • \\f denotes the Frobenius norm of a matrix, and 
tr(') denotes the trace of a matrix. Here, it is worth not¬ 
ing that the above problem formulation actually imposes both 
vertical and horizontal constraint propagation upon the ini¬ 
tial matrix Zu. That is, each column (or row) of Zu can 
be regraded as the initial configuration of a two-class label 
propagation proble m, which is formulated just the same as 
| |Zhou et al. , 2004| . Moreover, in this paper, we assume that 
the vertical and horizontal constraint propagation have the 
same importance for constrained image segmentation. 

The objective function given by Eq. dU) is further discussed 
as follows. The first and second terms are related to the ver¬ 
tical constraint propagation, while the third and fourth terms 
are related to the horizontal constraint propagation. The fifth 
term then ensures that the solutions of these types of con¬ 
straint propagation are as approximate as possible. Let F* 
and F^ be the best solutions of vertical and horizontal con¬ 
straint propagation, respectively. The best solution of our se¬ 
lective constraint propagation is defined as: 

f: = {f:+F*^)/2. ( 5 ) 

As for the second and fourth terms, they are known as 
Laplacian regulariz ation | |Zhu et al, 2003 [ |Zhou et al , 2004t 
[Belkin et al, 2QQ6| , which means that Fy and should not 
change too much between similar pixels. Such Laplacian reg¬ 
ularization has been widely used for different graph-based 
learning problems in the literature. 

To apply our selective constraint propagation (SCP) to con¬ 
strained image segmentation, we have to concern the follow¬ 
ing key problem: how to solve Eq. (0) efficiently. Fortunately, 


due to the problem formulation from a graph-based learning 
viewpoint, we can develop an efficient SCP algorithm using 
the label propagation technique i jZhou et al. , 2QQ4| | based on 
k-NN graph over Pu. The proposed SCP algorithm will be 
elaborated in the next subsection. 

2.2 Efficient SCP Algorithm 

Let Q{Fy,Fh) denote the objective function in Eq. 0. 
The alternate optimization technique can be adopted to 
solve miiiF^^Fh Q{^v^ Fh) as follows: 1) Fix Fh = 
and perform the vertical propagation by F* = 
arg miiiF^ Q{Fy,F^) \ 2) Fix Fy = F*, and perform the hor¬ 
izontal propagation by F^ = arg miui?^ Q(F*, F/^,). 

Vertical Propagation: When is fixed at F^, the solution 
of mini?^ Q{Fy^F^) can be found by solving 

- Zu) + 2,iLuFu + 2FFu - F^) = 0, 

U-t y 

which is actually equivalent to 

(/ + fiLu)Fy = (1 - P)Zu + /3F^*, ( 6 ) 

where /i = /i/(l + 7 ) and /3 = 7/(1 + 7 ). Since I -f jlLu is 
positive definite, the above linear equation has a solution: 

F: = {I + fiLu)-\{l-l3)Zu+l3F^). (7) 

However, this analytical solution is not efficient at all for con¬ 
strained image segmentation, since the matrix inverse incurs 
a large time cost of 0{N^). In fact, this solution can also 
be efficiently found using the label propagation technique 
I jZhou et al. , 2004| j based on k-NN graph over Pu (see the 
SCP algorithm outlined below). 

Horizontal Propagation: When Fy is fixed at F*, the solu¬ 
tion of mini?^ Q{F*,Fh) can be found by solving 

= 2{Fh - Zu) + 2iiFhLu + 2FFh - A;) = 0, 

(d-th 

which is actually equivalent to 

Fh{I + fiLu) = (1 - l3)Zu + /3F;. (8) 

This linear equation can als o be efficiently solv ed using the 
label propagation technique I jZhou et al. , 2^004| based on k- 
NN graph, similar to what we do for Eq. ® . 

Since the wight matrix Wu over Pu is derived from the 
original weight matrix W of the k-NN graph constructed 
over all the pixels according to Eq. o, Wu can be re¬ 
garded as the weight matrix of a k-NN graph constructed 
over Pu. Hence, we can adopt the label propagation tech¬ 
nique | |Zhou et al. , 2QQ4| | to efficiently solve both Eq. (| 6 ]) and 
Eq. ([ 8 ]). Moreover, to speed up our selective constraint prop¬ 
agation, we also discard those less important propagated con¬ 
straints during both vertical and horizontal propagation. That 
is, the two matrices Fy and are forced to become sparser 
and thus less computation load is needed during iteration. 

The complete SCP algorithm is outlined as follows: 

( 1 ) Compute Su = I — Lu, where Lu is given by Eq. (0; 

(2) Initialize Fy{0) = 0, F^ = 0, and F/j,(0) = 0; 


















(3) Discard those less important propagated constraints with 

Fy (t) < e during the vertical propagation, where we set 
e = 10 “^ in this paper; 

(4) Fy{tFl) = aSuFy{t)F{l-a){{l-P)ZuFf3F^),whQYQ 

Of = /i/(l + /i) G ( 0 , 1 ) and /3 = 7/(1 + 7 ) G ( 0 , 1 ); 

(5) Iterate Steps (3)-(4) for the vertical propagation until 

convergence at F*; 

(6) Discard those less important propagated constraints with 

Fh{t) < e during the horizontal propagation; 

( 7 ) Fh{t + 1) = aFh{t)Su + (1 - c.)((l - + /3F*); 

( 8 ) Iterate Steps (6)-(7) for the horizontal propagation until 

until convergence at F ^; 

(9) Iterate Steps (3)-(8) until the stopping condition is satis¬ 

fied, and output the solution F* = (F* + F^)/2. 

Similar to | |Zhou et al, 2004| |, the iteration in Step (4) con- 
verges to F* = (1 - a){I - Q!S'„)“^((1 - /3)Z„ + jSF^), 
which is equal to the solution (|7]) given that a = /i/(l + /i) 
and Su = I — Lu. Moreover, in our later experiments, we find 
that the iterations in Steps (5), ( 8 ) and (9) generally converge 
in very limited steps (<10). Finally, based on k-NN graph, 
our SCP algorithm has a time cost of 0{kN^) (Nu N). 
Hence, it can be considered to provide an efficient solution 
(note that even a simple assignment operation on F* incurs a 
time cost of 0{N^)). 

3 Constrained Image Segmentation 

In this section, we discuss how to exploit the selectively prop¬ 
agated constraints stored in the output F* of our SCP algo¬ 
rithm for image segmentation based on normalized cuts. The 
basic idea is to adjust the original weight matrix Wu over 
the selected subset of pixels Pu using these selectively prop¬ 
agated constraints. The problem of such weight adjustment 
for normalized cuts can be formulated as: 

min i||lV.-KllF + A||W„-W„||i, (9) 

Wu>0 ^ 

where Wu G is the new weight matrix over Py, and 

A is the regularization parameter. It is worth noting that the 
new weight matrix Wy is actually derived as a nonnegative 
fusion of F* and Wy by solving the above Fi-minimization 
problem. More notably, the Li-norm regularization term 
11 Wy — Wy 111 can force the new weight matrix Wy not only 
to approach Wy but also to become as sparse as Wy, given 
that Wy can be regarded as the weight matrix (thus sparse) of 
a k-NN graph constructed over Py. 

The problem given by Eq. Q can be solved based on some 
basic Fi-minimiza tion techniques ||Elad and Aharon, 2QQ6t 

[Mairal et al, 2008t[Wright et al, 2009[|Xiao et al,20\\\ . In 

fact, it has an explicit solution: 

if: =soft_thr(F:,IE,,A), (10) 

where soft_thr(-, •, A) is a soft-thresholding function. Here, 
we directly define 2 : = soft_thr(x, A) as: 

^^{zi=Ta.&x.{x-\,y), A </2 

\ 2:2 = max(0, min(x + A, y)), A > A ’ 


where A = ~ + 2X\zi — y\ and A = {z 2 — xV + 

2 A| 2;2 — y\. Since the Li-minimization problem given by 
Eq. ^ is limited to Py, finding the best new weight matrix 
W* incurs a time cost of 0{N^) (Ny ^ N). 

Once we have found the best new weight matrix IF* = 
xNu the selected subset of pixels Py, 
we can derive the new weight matrix W = {w{i, j)}NxN 
over all the pixels from the original weight matrix W = 
{w{i, j)}NxN of the k-NN graph as: 

wlii'J'), i,j e Pu,Puii') = i,Puij') = j 

otherwise 

where Py{i') denotes the i'-th member of Py. This new 
weight matrix W over all the pixels is then applied to nor¬ 
malized cuts for image segmentation. 

The full algorithm for constrained image segmentation can 
be summarized as follows: 

( 1 ) Generate the selectively propagated constraints using our 

SCP algorithm proposed in Section 2; 

(2) Adjust the original weight matrix by exploiting the selec¬ 

tively propagated constraints according to Eq. (fTOl) : 

(3) Perform normalized cuts with the adjusted new weight 

matrix for image segmentation. 

As we have mentioned. Steps (1) and (2) incur a time cost 
of 0{kN^) and 0{N^) (Ny <C N), respectively. Moreover, 
since the adjusted new weight matrix W is very sparse. Step 
(3) can be performed efficiently. Here, it is worth noting that 
the most time-consuming component of normalized cuts (i.e., 
eigenvalue decomposition) has a linear time cost when the 
weight matrix is very sparse. In summary, our algorithm runs 
very efficiently for constrained image segmentation. 

4 Experimental Results 

In this section, our algorithm is evaluated in the task of con¬ 
strained image segmentation. We first describe the experi¬ 
mental setup, including information of the feature extraction 
and the implementation details. Moreover, we compare our 
algorithm with other closely related methods. 

4.1 Experimental Setup 

Eor segmentation evaluation, we select 50 images from the 
Berkeley segmentation database | |Martin et al, 200 1| (along 
with ground truth segmentations), and some sample images 
are shown in Eigures [2] and [H It can be observed that these 
selected images generally have confusing backgrounds, such 
as the penguin and kangaroo images. Eurthermore, we con¬ 
sider a 6 -dimensional vector of color and texture features for 
each pixel of an image just as | |Carson et al, 2002] . The 
three color features are the coordinates in the L*a*b* color 
space, which are smoothed to avoid over-segmentation aris¬ 
ing from local color variations due to texture. The three tex¬ 
ture features are contrast, anisotropy, and polarity, which are 
extracted at an automatically selected scale. 

The segmentation results are measured by the adjusted 
Rand (AR) index |Hubert and Arabie, 1985| which takes val¬ 
ues between -1 and 1, and a higher AR score indicates 
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Figure 2: The results of constrained image segmentation. A small set of labeled pixels (object and background denoted by 
markers ‘o’ and ‘x’, respectively) are provided to infer the initial set of linear equality constraints and pairwise constraints. 


Table 1: The average segmentation results achieved by our 
NCutS-SCP algorithm with a varied number (i.e. Ns) of pix¬ 
els being selected randomly. 


Ne 

1,000 

1,500 

2,000 

2,500 

3,000 

3,500 

AR index 

0.47 

0.48 

0.48 

0.50 

0.50 

0.50 

Time (sec.) 

31 

32 

33 

35 

36 

37 


that a higher percentage of pixel pairs in a test segmenta¬ 
tion have the same relationship (joined or separated) as in 
each ground truth segmentation. We do not consider the 
original Rand index | |Rand, 197H |Unnikrishnan et al, 2007| 
for segmentation evaluation, since there exists a problem 
with this measure [Hubert and Arabie, 1985| . In the fol¬ 
lowing, our normalized cuts with selective constraint prop¬ 
agation (NCuts.SCP) is compared with three closely related 
methods: normalized cuts with linear equality constraints 
(NCuts_LEC) I Yu and Shi, 2004|, normalized c uts with spec¬ 
tral learning (NCuts_SL) | |Kamvar a/., 2003|, a nd standard 
normalized cuts (NCuts) | |Shi and Malik, 2000| . Here, we 
do not make comparison to other constraint propagation 
methods ||Lu and Carreira-Perpinan, 2QQ8t |Li et al, 2QQ8i 
Lu and Ip, 2010t|Lu and Peng, 2013| , since they have a poly¬ 
nomial time complexity and are not suitable for image seg¬ 
mentation. 

We randomly select a small set of labeled pixels to infer the 
initial set of linear equality constraints (for NCuts_LEC) and 
pairwise constraints (for NCuts_SCP and NCuts_SL). More- 


Original constraints Propagated constraints 


Figure 3: Illustration of the selectively propagated constraints 
obtained by our NCuts.SCP algorithm with = 2,500. The 
pixels within object and background are marked by blue ‘o’ 
and yellow ‘x’, respectively. 

over, we set the parameters of our NCuts_SCP algorithm as: 
k = 60,a = 0.9, f3 = 0.1, e = 10“^ and A = 0.001. The 
parameters of other closely related methods are also set their 
respective optimal values. 

4.2 Segmentation Results 

We first show the effect of Ng (i.e. the number of randomly 
selected pixels) on the performance of our NCuts.SCP al¬ 
gorithm in Table [TJ Here, we measure the performance of 
our NCutS-SCP algorithm for constrained image segmenta¬ 
tion by both AR index and running time averaged over all 
the images. In particular, we collect the running time by run¬ 
ning our NCutS-SCP algorithm (Matlab code) on a computer 
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Figure 4: The results of constrained image segmentation (cont.). A small set of labeled pixels (object and background denoted 
by markers ‘o’ and ‘x’, respectively) are provided to infer the initial set of linear equality constraints and pairwise constraints. 


Table 2: The average segmentation results achieved by differ¬ 
ent image segmentation methods. 


Methods 

NCuts NCuts SL NCuts LEC NCuts.SCP 

AR index 
Time (sec.) 

0.36 0.39 0.40 0.50 

28 31 25 35 


with 3GHz CPU and 32GB RAM. From Table [B it can be 
clearly observed that our NCuts_SCP algorithm requires more 
running time but leads to higher AR index when Ns takes a 
larger value. Considering the tradeoff between the effective¬ 
ness and efficiency of constrained image segmentation, we 
select Ns = 2, 500 for our NCuts_SCP algorithm. This set¬ 
ting is used throughout the following experiments. 

We further illustrate the selectively propagated constraints 
obtained by our NCuts.SCP algorithm with Ns = 2, 500 in 
Figure [51 Here, to explicitly represent the selectively propa¬ 
gated constraints, we need to infer the labels of Ns randomly 
selected pixels from them. In fact, this inference can be done 
by simple voting according to the output of our SCP with the 
initial set of labeled pixels being regarded as the voters. Once 
we have inferred the labels of Ns randomly selected pixels, 
we can show them out by marking the pixels within object and 
background by blue ‘o’ and yellow ‘x’, respectively. From 
Figure [51 we find that the selectively propagated constraints 
obtained by our NCuts.SCP algorithm are mostly consistent 
with the ground truth segmentation. 

The comparison between different image segmentation 


methods is listed Table [21 Meanwhile, this comparison is 
also illustrated in Figures [2] and [H Here, the segmentation 
results are evaluated by both AR index and running time av¬ 
eraged over all the images. In particular, we collect the run¬ 
ning time by running all the algorithms (Matlab code) on a 
computer with 3GHz CPU and 32GB RAM. The immedi¬ 
ate observation is that our NCuts_SCP algorithm significantly 
outperforms the other three methods in terms of AR index. 
Since our NCuts.SCP algorithm incurs a time cost compara¬ 
ble to closely related methods, it is preferred for constrained 
image segmentation by an overall consideration. In addition, 
it can be clearly observed that NCuts.SCP, NCuts_LEC, and 
NCuts.SL lead to better results than the standard NCuts due 
to the use of constraints for image segmentation. 

5 Conclusions 

In this paper, we have investigated the challenging problem 
of pairwise constraint propagation in constrained image seg¬ 
mentation. Considering the local homogeneousness of a nat¬ 
ural image, we choose to perform pairwise constraint prop¬ 
agation only over a selected subset of pixels. Moreover, we 
solve such selective constraint propagation problem by devel¬ 
oping an efficient graph-based learning algorithm. Finally, 
the selectively propagated constraints are used to adjust the 
weight matrix based on Li-minimization for image segmen¬ 
tation. The experimental results have shown the promising 
performance of the proposed algorithm for constrained im¬ 
age segmentation. For future work, we will extend the pro- 































































posed algorithm to other challenging tasks such as semantic 

segmentation and multi-face tracking. 
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